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(54) Wide-angle tonafledewwiplnfl method and apparatus 

(57) A metiiod and apparatus far transforming wide angle video Into perspective correrted viewing zones 
SJU eSSr a sSe J^r or multiple usen. may select, orient and magnify. TJe Pr««r i"::^'" J"*^ 
Mtrtures a wide angle digital video Input by any sutebls means. The captured image is then stored m a 
Sab?e iJemory me!.» 80^^ of the image may be selected at a later time. When a portion of the stored 
So s reLrte?Sr iewing' a plurality of discrete viewing vectors In three ^'^'^^"'^"S^tir.^l'^r 
Sendee Input and transfomied to a plurality of control points In a tv« dimensional Pte"«f ^«"V rther 
suitable surface. Tlie area between these points which is still warped from the origmal wide angle image 
caStu^e is tS^^rSformed to a perspecti^; corrected field of view. ^^^J'^^^^'L'^'i^'''' " 
then displayed on a suitable displaying apparatus, such as a monitor or head mounted display. 
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WIDE-ANGLE IMAGE DEWARPING METHOD AND APPARATUS 



The present invention relates to a method and apparatus for displaying a 
perspective corrected field of view from wide angle >rtdeo sources, and more particularly 
relates to permitting the user of an orientation sensing means to view a selected portion 
of stored or real time video encoded from a wide angle source and transforming that 
portion to a perspective-corrected field of view. 

Background of the Invention 
"Virtual reality"' and "telepresence" have become extremely popular for use in 
research, industrial and entenainmenl applications. In "virtual reality ", or VR. a user 
IS permitted to view a computer-generated graphical representation of a selected 
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environment Depending on the sophistication of the hardware and software used to 
generate the virtual reality environment the user may be treated to a three dimensional 
view of the simulated environment In "telepresence." a user is pemiltted to view a real- 
wortd, live or recorded environment from a three dimensional perspective. 

In addition, in some higher end systems the user Is permitted to see different 
portions of the VR and telepresence environments simply by moving or orienting his 
head In one or more degrees of freedom. This permits the user to obtain the sensation 
that he is immersed in the computer-generated/real-world environment High end 
devices detect pan. roll .and tilt motions by the user and cause the environment to 
change accordingly. The panVtiltVroll motions may be inputted by many types of input 
devices, such as joysticks, buttons or head orientation sensors (which may be 
connected to head mounted displays). 

in VR applications, a continuing problem is how to render a three dimensional 
environment of the quality and speed users want while offering the product at a price 
they can afford. To make a realistic envirorvnent, sCich as in a three dimensional video 
game, many tiiree dimensional polygons need to be rendered. This rendering requires 
prohibitively expensive hardware which greatly restricts ttie commerdai value of such 
a system. 

In relation to telepresence applications, a continuing problem with the prior art - 
is how to encode sufficient data that a viewer may arbitrarily move his viewing 
perspective within the telepresence environment and not look beyond the field of view. 
One relatively simple solution, where the telepresence environment is based on a real 
three dimensional environment, is to simply use the head orientation sensors to cause 
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a camera to track the orientation of the viewer. This has obvious limitations In that only 
one viewer can be in the telepresence environment at a time (since the. camera can 
only track one viewer, and the other viewers will not typically be able to follow the head 
motions of the controlling viewer) and, also, prerecorded data cannot be used. 
Further, there Is an inherent delay between a change in user viewing perspective and 
the time that it takes to realign the corresponding camera. These limitations greatly 
restrict the value of such systems. 

One method for overcoming each of these limitations is to encode, either in real 
time or by pre-reoording, a field of view largely equivalent to the entire range of motion 
vision of a viewer — that is, what the viewer would see if he moved his head in each 
permitted direction throughout the entire permissible range. For example, encoding 
substantially a full hemisphere of visual Information would permit a plurality of viewers 
a reasonable degree of freedom to interactively look in a range of directions within the 
telepresence environment 

The difficulty with this approach is that most means for encoding sudi 
information distort, or warp, the visual data, so that the information must be corected. 
or "de-warped" before a viewer can readily assimilate it For example, a typical 
approach for encoding substantially a fUll hemisphere of infonnation involves using a 
fish-eye lens. Fish-eye lenses, by their nature, convert a three dimensional scene to 
a two-dimensional representation by compressing the data at the periphery of the field 
of view. For the information to be viewed comfortably by a viewer in the VR 
environment, the visual data must be decompressed, or dewarped, so that it is 
presented in normal perspective as a two dimensional representation. 
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One solution to the distortion problem is proposed in U-S. Patent 5,185,667 
issued to Steven Zimmerman. The '667 patent describes an apparatus ydnlch effects 
camera control for pan, tilt, rotate and zoom while having no moving parts. Through the 
use of a fisheye lens and a complicated trigonometric technique, portions of the video 
images can be dewarped. However, the solution proposed by the *667 patent is 
impractical because it is insufficientiy flexible to accommodate the use of other lenses 
besides a theoretically perfect hemispherical fisheye lens without the introduction of 
mathematical errors due to the misfit between the theoretical and the actual lens 
characteristics. This solution also introduces undesirable trigonometric complexity 
which slows down the transformation and is overly expensive to Implement This 
solution further maps each irxiividual pixel through the complex trigonometric mapping 
formula further reducing the speed of the transformation from one coordinate system 
to another. 

As a result, there has been a substantial need for a method and apparatus which 
can dewarp encoded wide angle visual data with suffiaent speed and accuracy to 
permit a viewer to immerse himself in a VR or telepresence environment and look 
around wifBin tha environment while at the same time permitting other viewers 
concurrently to independently errgage in the same activity on ttie same broadcast video 
signal There has also been a need for a method and apparatus capable of providing 
such dewarping on a general purpose high speed computer. 

Summary of the Invention 

The present invention overcomes the limitations of the prior art In particular, 
the present invention transforms a plurality of viewing vectors within a selected portion 
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of the wide angle, three dimensional video input Into two dimensional control points and 
uses a comparatively simple method to transform the image between the control points 
to create a perspective-corrected field of view. 

More specifically, the present invention Is drawn to a method and apparatus 
which provides perspective corrected views of live, prerecorded or simulated wide 
angle environments. The present invention first captures a wide angle digital video 
input by any suitable means, such as through the combination of a high resolution 
video camera, hemispherical fisheye lens and real time digital image capture board. 
The captured image is then stored in a suitable memory means so portions of the 
image may be selected at a later time. 

When a portion of the stored video is selected, a plurality of discrete viewing 
vectors in three dimensional space are chosen and transformed into a plurality of 
control points in a comesponding two dimensional plane. The area between the control 
points, which is still warped from the original wide angle image capture, is then 
transfbmied into a perspective conected field of vieWJlhrough a biquadratic polynomial 
mapping technique. The perspective conned field of view is then displayed on a 
suitable displaying apparatus, such as a monitor or head mounted display. The present 
invention further has the ability to sense an Inputted selection, orientation and 
magnification of a new portion of the stored video for transformation. 

In comparison with the prior art, the present invention provides a dependable, 
low cost, faster and more elegantly simple solution to dewarping wide angle three 
dimensional images. The present invention also allows for simultaneous, dynamic 
transformation of wide angle video to multiple viewers and provides each user with the 
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ability to access and manipulate the same or different portions of the video input in 
VR applications, the present invention also allows the computer generated three 
dimensional polygons to be rendered in advance; thus, users may view the 
environments from any orientation quickly and without expensive rendering hardware. 

It is therefbre one otjjed of the present invention to provide a method and 
apparatus for dewarping wide angle video to a perspective corrected field of view which 
can then be displayed. 

It is another object of the present invention to provide a method and apparatus 
which can simultaneously transform the same or different portions of wide angle video 
input for different users. 

it is yet another object of the present invention to provide a method and 
apparatus which allows selection and orientation of any portion of the video input 

It is still another object of the present invention to provide a method and 
apparatus for magnification of the video input 

it is still another object of the present in'tontian to provide a method and 
apparatus which performs all of the foregoing objects while having no moving parts. 

These and other objects of the invention will be better understood from the 
following Detailed Description of the Invention, taicen together with the. attached 
Figures. 

THE FIGURES 

Figure 1 shows a functional block diagram of one embodiment of the present 

invention. 

Figure 2 diagrams the geometry between three dimensional (X-Y-Z) space 
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and its cx>rresponcllng two dimensional (U-V) plane. 

Figure 3a shows a bilinear mapping of a warped Image. 

Figure 3b shows a biquadratic mapping of a warped Image. 

Figure 4 shows a side view of a viewing vector from a three dimensional (X- 
Y-Z) wide angle lens as it is seen on a two dimensional (U-V) plane. 

Figure 5 shows a three dimensional field of view along with a plurality of 
viewing vectors according to the present invention. 

Figure 6 shows a block diagram of the elements of a forward texture mapping 
ASIC according to the present Invention. 

Figure 7 shows an example of a U-V source texture transfonmed into a X-Y 
plane destination texture according to the present invention. 

Rgure 8 is one embodiment of how to obtain a 360 degree view using six 
hemispherical fisheye lenses according to the present Inventloa 

Figure 9 is a functional flow chart of one embodiment of the present 
invention. 

Detailed Description of the Invention 
Referring now to Figure 1, an embodiment of the present invention for 
processing wide angle video information in real time is shown. A high resolution video 
camera 10 having a wide angle lens 20, such as a hemispherical fisheye lens, is 
directed to a real world scene 2Z The output 24 of the camera 1 0 is provided to a real 
time image digitizing board 30, commonly referred to as a 'frame grabber," located in 
or operatively connected to a conventional high speed computer indicated generally at 
1 50. The camera 1 0 may be any camera which is capable of using a wide angle lens 
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and providing suitable «so.udon.», most ln«anc« the e««awi,,b.avld«. camera, 

a,«,ough in some Instances .t may be desirable to use a s«.. ftame came™. One 
suitable™ lens Is the Na*or AuU, 1 :1.8 f=amm «sheye lens «-hlch can be adapted 
a standard high resoluUon broadcast video camera v,im e NIKon FW^ng TMW-B1 
converter. The computer ISO Is any computer capable of receiving and processing 
|nft,m,ation at an acceptable rat. and may. for e«mpla. be an 80486-based or 
Pentium^-based sy«em. or other computer platfom, such as are m«.e by Silicon 
Graphics, sun Micm Systems. Apple Computer. -.or similar o«,er computer 
manufacturers. 

The (isheye lens 20 causes the video output signal 24 f«m the camera 10 to be 
op.l«l.yv«,p«lir,anon*ear manner. Betor. the Image can be comfortably viewed. 
^ , user, perspeouve^otlng measure, must be t-«n. The digitized video signal 
24 I. thus transferred through the digitizing bo««. 30 (typically but not nec..s»lly 
operating at 30 frames per second) into memory 40 of the computer 160 so that 
portions Of the Video plctu« c«n be randomly ed^ed by a microprocessor SO. also 
within the computer ISO, at any time. 

■me dewarping software Is also sto«J In memory 40 and Is epplied to the video 
signal 24 by the microprocessor SO. The stored video signal Is th«. transmitted from 
„,™ry 40 to a special purpose ASIC 60 capable of biquadratic or higher order 
polynomial translOrmeBons for texture warping and interpolatlcn. AltemaUvely, me 
.eKture warping ASIC 60 may be omitted and its functionality may be perfom-ed by 
software. Phantom lines have been used to show the optional nature of ASIC 60. The 
perspecBve correoed video s«nal Is next transmitted to a video output stage 70, such 
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as a standard VGA card, and from there displayed on a suitable monitor, head mounted 
display or the like 80. An input device 90. such as a joystick or headtracker (which 
senses the head movements of a user wearing a headmounted display), transmits 
position information through a suitable input port 100, such as a standard serial, 
parallel or game port, to the microprocessor 60 to control the portion of the stored video 
that is selected, dewarped and displayed. The input device 90 also transmits 
nsil/pitch/yaw information to the microprocessor 50 so that a user may control the 
orientation of the dewaiped video signal. Further, one skilled in the art will appreciate 
that a magnification option could be added to the Input device 90 to allow the user to 
magnily the selected portion of video input, constrained only by the resolution of the 
camera 10. 

Rgure 2 shows a real world three dimensional environment 200 which has been 
imaged by the wide angle lens 20. This environment is defined by the Cartesian 
coordinate system in X. Y and Z with the viewpoint defined to be the origin of the 
coordinate system. The viewing direction of the u'ser, as defined by the input device 
90, is given as a viewing vector in the X-Y-2 coordinate system. The Image plane 21 0 
containing the warped wide angle image is defined by a two dimensional coordinate 
system in U and V, with the origin of the coordinate system coincident with the origin 
of the X-Y-Z coordinate system. If the field of view of the lens 20 is sufficient, and the 
lens is rotationally symmetric about the viewing axis, the digitized warped image will be 
roughly circular in the U-V plane. 

The first generation of ASICs, developed for low-cost texture mapping of three 
dimensional graphics, mapped video Images through a bilinear technique, such as is 
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shown in Figure 3(a). These chips were able to apply linear interpolation to texture 
pixels in both the X and Y directions and could thereby stretch rectangular source 
textures to any two dimensional quadrilateral shape. An example of a chip of this type 
is the Artist Graphics 3GA chip. These bilinear chips do, however, introduce texture 
errors for polygons whose vertices have been subject to significant amounts of 
perspective, and further are not capable of sufficiently high order texture distortion to 
adequately flatten extreme wide angle views, such as with hemispherical fisheye 
lenses. 

Figure 3b shows an example of a biquadratic technique, such as is now coming 
onto the market The preferred embodiment of the present Invention uses an ASIC chip 
which implements a texture warping technique of at least second polynomial order. 
The present invention is of sufficient simplicity that this technique could also be 
implemented in software on a general purpose high speed computer, such as a Silicon 
Graphics Indigo™ computer or a Pentium™ based computer. 

The warped image in the i>V plane, shown'in Figure 2, has a radius 220 equal 
to RADIUS pixels with an origin at UORIGIN and VORIGIN. For any rotationally 
symmetric lens, the warping effect of the lens can be described by a single lens 
equation, r=^6), where the function f(6) maps any incident ray at angle 6 from the axis 
of viewing to a radial displacement in pixels, r, from the center of the U-V plane, as 
shown in Figure 4. 

For any given viewing direction in three dimensional X-Y-Z space, we then have: 



11 



P-1757 



z 



(1) 




s 



In the case of an fdeal hemispheric fisheye lens, f(e) « (RADIUS)(sln(e)) and the lens 
equation which results is: 



Equations (T) convert an Inputted X-Y-Z three dimensional viewing vector into a 
corresponding control point in the U-V plane. 

To dewatp a rectangular portion of the wide anjie video Input for a given viewing 
direction (x,y,z), eight other viewing vectors, which sunxDund the primary viewing vector, 
are computed at the field of view angles fov_h and fbv.v from the primary viewing 
vector, as shown In Rgure 6. Each of these nine vectors are then projected ftam three 
dimensional X-Y-Z space to the two dimensional U-V plane by equations (1). The 
result is a 3x3 grid of control points in the U-V plane, with the edges of the grid curving 
mathematically to confbmi with the curvature induced by the warping effect of the wide 
angle lens. 

The global blvariate polynomial transformation: 




(2) . 
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{ -0 /-o with X and y now 

referrino to the pM 

t ^ y coordinates in the 

^■2^ 2- ^y* > output device 

1*0 y«o 



is then found to describe the geometric correction necessary to transform the region 
within the warped 3x3 grid in the U-V plane into a perspective corrected field of view. 
A biquadratic potynomial transformation. N^Z in the above equations, has been 
selected because a second order polynomial approximates the warping characteristics 
of most lenses to an adequately high degree of precision and because there is existing 
hardware to perfom:! the resulting biquadratic transformation. However, it will be 
appreciated, by one skilled in the art that other polynomial transformations of higher 
degree could be used to increase the precision of the transformation. 
Expanding the above equations (3): 

ft 

I *j X^^ Xj X, 

I «7 >7 Xj 

"9] 

[l X, x^, xp x. 

The values for v and b^ can be similariy found. In matrix fomn. the expanded equations 
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(4) can be written as: 

To discover a^ and t), according to tlie method of the present invention, a pseudo- 
inverse technique is used. However, one skilled in the art will appreciate that there are 
methods to solve equations (5) other than by a pseudo inverse technique, i.e. a least 
squares technique. The pseudo-Inverse solutions for A and B in the above equation 

(5) are: 

Therefore, for a target display of a given pixel resolution NxM, W and its pseudo- 
Inverse QtfWf^W can be calculated a priori. The values for a, and b, are then found 
by mapping the points in the U-V plane for the 3x3 grid of control points using the 
above equations (6). The biquadratic polynomial transformations of the equations (3) 
are then used to transform the area between the control points. In this embodiment, 
the detemiination of the coordinates of each pb(el In the U-V plane takes a total of 
thirteen multiplication and ten addition operations. Additionally, three of the required 
multiplication operations per pixel may be obviated by storing a table of xy, x* and y* 
values for each xy coordinate pair in the dewarped destination image. In another 
embodiment, the "x" values which do not vary as "y" changes (i.e. a, " x + a4" x^ and 
bi * x + b^ " X*) may also be precomputed and stored. Likewise, the y values which 
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do not vary as changes may be precomputed and stored These further 
optimizations reduce the operations needed to determine the coordinates of each pixel 
in the U-V plane to two multiplication and four addition operations. 

It wilt be appreciated by one skilled in the art that the accuracy of the dewarping 
transformation will increase as the number of transformed viewing vectors increases, 
i.e. a 4x4 grid of control points will produce a more accurate transfomistion than a 3x3 
grid of control points. However, the amount of increase in accuracy quickly draws near 
an asymptote as the number of control points is increased. One skilled in the art will 
recognize, therefore, that there is little reason in increasing the number of viewing 
vectors to more than half of the total number of pixels in the displayed regioa 

It will be further appreciated by one skilled in the art that the selection of a 
rectangular shape of the video input could be changed to another shape and still be 
within the scope of the inventloa Further, the number of control points could be 
increased or decreased to correspondingly increase or decrease the accural of the 
transfonmatioa Further still, an image filtering stage could be applied during the 
inverse texture mapping without deviating fr^m the present inventioh. 

Rgure 9 shows a functional flow chart of the major elements of one embodiment 
of the present inventton. Rrst, the fixed warped image parameters (step 400) are 
defined, such as the size of the input image, the input image radius, and the input 
image center in U-V coordinates typically measured in pixels. The next step 410 is to 
initialize the variable dewarped image parameters, such as the size of the dewarped 
image area, the horizontal and vertical field of views (generally shown in degrees), the 
creation of an untransformed view cone centered in this embodiment on the +Z axis 
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and the initialization of the layout and number control points used therewith. Typically, 
the next step is to load the precomputed inner-loop matrix values as weli'as the "x/* 
product temis, as shown In step 420, to ensure that the transformation is accomplished 
as quickly and efficiently as possible. In step 430, the video signal is input to the 
system in any suitable form, i.e. live or pre-recorded real-time digitized video or 
computer synthesized video environments. The system then allows the user to select 
the viewing vector (step 440) which in turn determines the portion of video which is to 
be transformed. The control points are next transformed from the selected viewing 
vectors (step 450) and the region defined by the control points is dewarped (step 460). 
The signal is then sent to the video buffer and to an appropriate viewing apparatus 
(step 470). There is a recursive loop from step 470 to step 430 to allow the video 
signal to tse refreshed, as is needed with live motion video. The loop also allows the 
user to make on-the-fly selections of alternate portions of the incoming video. 

The foregoing description describes an inverse texture mapping technique 
whereby the bkjuadratic output (X-Y) is mapped to the input (U-V). In the case where 
a forward texture mapping ASIC is used, the coordinates for the destination control 
points in must be supplied so that the rectilinear source tacture region can be 
mapped from the U-V plane, as provided by the inverse texture mapping software 
solution above, to the X-Y plane. An example of a forward texture mapping ASIC Is the 
NV-1 chip sold by N-Vidia Corporation. Figure 6 gives another example of an ASIC 
chip 230 which accepts control pixel coordinates at a forward mapping solution stage 
240. Four pixel coordinates will be accepted for a bilinear mapping, nine in the case 
of quadratic mapping. 16 in the case of cubic mapping, etc. These control pixels are 
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produced In the host CPU according to the equations (7), below. As shown in Figure 
7. the coordinates of a rectangular bounding box are determined to enclose an 
exemplary 3x3 grid of control points u,v, in the U-V plane. The comers of the bounding 
box are found from the u^,extrBma Using the same technique described in equations 
(3). a, and b, for N=2 can be solved with the equations (7) for enclosing the region to 
be warped to the comers of the display screen. 

(7) 

Thus] the same control points for the U-V plane map to the comers of the display 
screen in the X-Y plariei The warped ^gions outside the bounding box may be clipped 
by hardware or software so that they are not visible on the display screen. • 

The source pixel coordinates, which are ffed from the host CPU, are converted 
to a, and b, coordinates for feiward mapping In the focward mapping solution stage 240, 
again using techniques mathematically equivalent to those of the equations (7). A 
series of instwctlons is flirther sent from the host CPU to the chip 230 and received by 
a control unit 260. The control unit 260 sequences and controls the operation of the 
other functional stages within the chip 230. The host CPU also directs a linear 
sequence of source pixels, which are to be warped, to an interpolation sampler stage 
250 within chip 230. Optionally, these can be subject to a low-pass spatial prefiltering 
stage 270 prior to transmission to the chip, to reduce sampling error during the warping 
process. Thus, within the chip 230, the source pixels and the a, and coordinates are 
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both fed to the Interpolation sampler 250. For each input pixel, one or more destination 

pixels together with their con^sponding X-Y destination coordinates ar^.produced. 

These warped pixels are then fed into the video frame buffer 280. located outside of the 

ASIC chip 230. Optionally, anti-aliasing circuitry 290 within the chip performs 

Interpolation on output pixel values, such as bilinear interpolation between adjacent 

pixel samples, to minimize the effects of output spatial quantization error. One skilled 

in the art will recognize tiiat the preceding hardware solution Is merely exemplary and 

that there are many such solutions which could be employed and still fall within the 

scope of tiie present InventiorL 

The techniques described herein may also be applied to synthetic images. Such 

images may be created entirely within a computer environment and may be composed 
of three dimensional geometrical descriptions of objects which can be produced by 
computer graphics rendering techniques generally known to those skilled in the art 
Typically, synthetic images are produced by linear perspective projection, emulating 
tine physical process of imaging onto planar film with a lens having a nanow field of 
view and produdng a view of the synthetto environment as seen through a cone or 
truncated three dimensional pyramid. The color, intensity shading and other simulated 
physical properties of each pixel on the planar image grid can also be readily 
determined. For a syntinetic environment, the viewing vectors in X-Y-Z space are 
rewritten in terms of Uie warped control points coordinates in the U-V plane. 
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A direction vector fn X-Y-Z space can thus be generated for each pixel In the U- 
V plane in the synthetic wide angle image which is created. For a perfect 
hemispherical fisheye, the generated vectors point in all directions within the created 
hemisphere, spaced to the JimitSLOfihs-resolution of the UA/ image. This simulates a 
non-planar image grid, such as the projection of the synthetic environment onto a 
surface of a spherical image substrate. In this way, a fisheye or other wide angle image 
of a synthetic three dimensional environment can be produced. This technique can be 
used for the production of three dimensional modeled cartoons or Interactive home 
gaming applications, among others. 

One sidlled in the art will appreciate that the present invention may be applied 
to a sequence of wide angle images changing in time, either live or recorded to an 
analog or digital storage media The image substrate for recordation may be an 
electronic two dimensional image sensor, such as a CCD chip, or photographic film 

capable of chemically recording the image for subsequent transfer into digital form. 
One skilled in the art will also appreciate that the present invention is not limited 

to transforming wide angle video onto a planar (U-V) surface, but that it is within the 

scope of tiie invention to transform wide angle video onto any suitable surface for 

displaying the video for the user 
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Further, tvwo real world, wide angle lenses can be positioned opposite each other 
to permit near 360 degrees of total coverage of the environment *:lf seamless 
omnidirectional coverage of an environment is required, this could be achieved with six 
wide angle lenses positioned along each direction of a three dimensional axis, as 
shown in Figure 8. This arrangement can be coupled with a video switching 
mechanism for choosing which signal is to be dewarped for the selected view and 
orientation of the video input 

Further still, the same video signal may be simultaneously transmitted to an 
arbitrarily large number of viewers all having the ability to simultaneously dewarp the 
same or different portions of the video input as in the case of interactive cable TV 
viewing or multiple player online interactive video game playing. 

Havingftdly described the pr^ierred embodiment of the present invention, it will 
be apparent to those of ordinary sidll in the art that numerous alternatives and 
equivalents exist which do not depart from the invention set forth above. It is therefore 
to be understood that the present invention Is hot to be limited by the foregoing 
description, but only by the appended claims. 
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What is claimed is: 

1. A method for providing perspective corrected views of live, prerecorded or 
simulated environments from wide angle video signals, the method comprising: 
capturing a wide angle video input; 
storing the captured video input; 

transforming a plurality of viewing vectors to a plurality of control points on a 
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portion of the video input; 

transforming an area between the plurality of control points to a perspective 
corrected field of view; 

displaying the perspective corrected field of view; 

sensing inputted infbnnation; and 

controlling the transformation and display of the perspective con-ected field of 
view through the inputted information. 

2. An apparatus for transforming wide angle video, signals from live, 
prerecorded or computer simulated environments into perspective corrected video 
signals, the apparatus comprising: 

video capturing means for capturing a wide angle video input; 
memory means electrically coupled to the video capturing means for storing 
the video input; 

processing means electrically coupled to the memory means fbr transforming 
a plurality of viewing vectors to a plurality of control points on a portion of the video 
input, the processing means further transforming an area between the plurality of 
control pointis to a perspective corrected field of view; 

output display means electrically coupled to the memory means fbr displaying 
the perspective conwted field of view; and 

selection means electrically coupled to the processing means for selecting 
portions of the wide angle video input for the processing means to transform. 

3. A method for correcting the perspective of wide angle video data 
comprising the steps of: 
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receiving video information comprising a wide angle view of a scene; 
pemnitUng the viewer to identify an area of interest within the wider angle view; 
digitizing at least a portion of the wide angle vi^, 
associating with the area of interest identified by the viewer a plurality of 
control points; 

transforming an area associated with the plurality of control points to a 
perspective corrected field of view; and 

providing the perspective corrected field of view lor display. 

4. The method of claim 3 In which the area of interest is the same as the 
area transformed. 

5. The method of claim 3 wherein the number of control points Is between 
five and one-half of the number of pixels in the output Image. 

6. The method of claim 3 wherein the number of control points is nine. 

7. The method of claim 3 wherein the video infomiation is taken from at least 
one of live video, stored video or computer generated video. 

♦ 

8. The method of daim 3 in which the transformation is performed by inverse 
texture mapping. 

9. The method of claim 3 in which the transformation is performed by forward 
texture mapping. 

10. The method of claim 3 in which the transformation is performed by 
biquadratic texture mapping. 

1 1 . The method of claim 3 in which the transformation Is performed by 
bicubic texture mapping. 



P-1757 

23 

12. A method for mapping data from a three dimensional region onto a 
bounded region of a plane, comprising the steps oft 

receiving at least a portion of a set of data having a region defined by a 
biquadratic equation; 

defining a first area of interest for mapping, the first area of interest being at 
least a portion of the set of data; 

associating with the defined first area a plurality of control points; and 

transforming a second area associated with the plurality of control points into 
a planar display* 

1 3. Apparatus for correcting the perspective of wide angle video, Including: 
means for receiving video information comprising a wide angle view of a 

scene; 

means for allowing the viewer to identify an area of interest within the wide 
angle view; 

means for digitizing at least a portion of the*-wide angle view; 

means for associating with the area of interest identified by the viewer a 
plurality of control points; 

means for transforming an area associated with the plurality of control points 
to a perspective corrected field of view; and 

means for providing the perspective corrected field of view for display, 

14. Apparatus for correcting the perspective of wide angle video data 
comprising: 

means for receiving and storing video information representing a wide angle 



P-1757 

view; 

input means for identifying a first area of interest among the vided 
information; 

control means responsive to the input means for associating with the area of 
interest a predetermined number of control points; and 

texture mapping means responsive to the control means for transforming a 
second area associated with the control points from the wide angle view into a 
predetermined second field of view. 

1 5. The method of daim 12 wherein the first area and second area are 
substantially congruent 

1 & The apparatus of daim 1 4 wherein the first area and second area are 

« 

substantially congruent 

17. A method substantially as herein described, with 
reference to the accompanying drawings. 

18. Apparatus substantially as herein described, with 
reference to the accompanying drawings 
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